1. Field of the Invention
The present invention relates to an image processing device, an image processing method and a computer-readable medium by which recognizes an augmented reality marker within a captured actual image and performs a superimposed display of a virtual object on the actual image.
2. Description of the Related Art
Mobile terminal devices, such as smartphones, are becoming increasingly multifunctional and generally have an image processing function referred to as Augmented Reality (AR), in addition to a camera function (imaging function). In this function, a three-dimensional virtual object (for example, a character image, etc.) is superimposed and displayed on a predetermined area within an image captured by the camera function. Specifically, when a photographic subject is captured by the imaging function in a state where an AR marker composed of printed material (a marker on which a white pattern is drawn within a square black frame) is arranged at or near the location of the photographic subject, the mobile terminal device recognizes the AR marker contained in the captured image by analyzing the captured image, and performs a superimposed display of a virtual object on an area of this AR marker. In this type of conventional technique where a superimposed display of a virtual object is performed on an area of an AR marker, the three-dimensional location and the direction of an AR marker are detected based on the size and the tilt of its pattern, and the display location and the display direction of a virtual object on a captured image are determined based thereon (refer to Japanese Patent Application Laid-open (Kokai) Publication No. 2005-234757).
In the recognition of an AR marker within a captured image, a predetermined direction (such as the front side) of the AR marker is determined based on a white pattern within a black frame constituting the AR marker, and the display direction of a virtual object is determined to be the front side of the AR marker. FIG. 6 is a diagram showing an example where a virtual object (character) is displayed so that its front side (face) is directed toward the same direction as the front side of an AR marker. The AR marker of this example is patterned so that a white area having a random shape is arranged within a square black frame in which the black area and the white area have a predetermined proportion.
In this instance, a mobile terminal device analyzes a captured image (actual image), focuses on the black frame portion of an AR marker in the captured image, and thereby detects the area of the AR marker (the area of the black frame). Subsequently, the mobile terminal device then identifies the pattern of the white area drawn within the AR marker, and thereby recognizes the AR marker. Then, the mobile terminal device determines the front side of the AR marker based on the pattern shape of the white area, and performs a superimposed display of a three-dimensional virtual object on the area of the AR marker within the image so that the virtual object is directed toward the same direction as the front side of the AR marker.
However, since rotation control and a superimposed display is performed so that the front side of the three-dimensional object is directed toward the same direction as the front side of the AR marker, for example, if the location of the photographer deviates from the front side of the AR marker and captures an image from the back side of the AR marker, a backward virtual object is displayed. Also, if the photographer captures an image from one side of the AR marker, a sideways virtual object is displayed. As a result of this configuration, when imaging from the front side of an AR marker is troublesome, the user can only view a backward virtual object or a sideways virtual object.
An object of the present invention is to enable easy display of a virtual object toward a direction desired by a user, when performing a superimposed display of the virtual object in a captured image containing an augmented reality marker.